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Abstract 

Background: There is increasing interest in the contribution of the quality of nursing care to patient outcomes. 
Due to different casemix and risk profiles, algorithms for administrative health data that identify nursing-sensitive 
outcomes in adult hospitalised patients may not be applicable to paediatric patients. The study purpose was to test 
adult algorithms in a paediatric hospital population and make amendments to increase the accuracy of 
identification of hospital acquired events. The study also aimed to determine whether the use of linked hospital 
records improved the likelihood of correctly identifying patient outcomes as nursing sensitive rather than being 
related to their pre-morbid conditions. 

Methods: Using algorithms developed by Needleman et al. (2001), proportions and rates of records that identified 
nursing-sensitive outcomes for pressure ulcers, pneumonia and surgical wound infections were determined from 
administrative hospitalisation data for all paediatric patients discharged from a tertiary paediatric hospital in Western 
Australia between July 1999 and June 2009. The effects of changes to inclusion and exclusion criteria for each 
algorithm on the calculated proportion or rate in the paediatric population were explored. Linked records were 
used to identify comorbid conditions that increased nursing-sensitive outcome risk. Rates were calculated using 
algorithms revised for paediatric patients. 

Results: Linked records of 129,719 hospital separations for 79,016 children were analysed. Identification of comorbid 
conditions was enhanced through access to prior and/or subsequent hospitalisation records (43% of children with 
pressure ulcers had a form of paralysis recorded only on a previous admission). Readmissions with a surgical wound 
infection were identified for 103 (4.8/1,000) surgical separations using linked data. After amendment of each 
algorithm for paediatric patients, rates of pressure ulcers and pneumonia reduced by 53% and 15% (from 1.3 to 0.6 
and from 9.1 to 7.7 per 10,000 patient days) respectively, and an 84% increase in the proportion of surgical wound 
infection (from 5.7 to 10.4 per 1,000 separations). 

Conclusions: Algorithms for nursing-sensitive outcomes used in adult populations have to be amended before 
application to paediatric populations. Using unlinked individual hospitalisation records to estimate rates of nursing- 
sensitive outcomes is likely to result in inaccurate rates. 



Background 

Nursing-sensitive outcomes are "changes in health status 
upon which nursing care has had a direct influence" 
[1,2] [p 1]. The concept arose from the quality improve- 
ment agenda of the Joint Commission on Accreditation 
of Health Care Organizations (JCAHO) in the United 
States (US) in the late 1980s [3]. More recently, concerns 
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about the changing nurse skill mix have led researchers 
to investigate associations between the incidence of 
nursing-sensitive outcomes and levels of nurse staffing 
[4-8]. 

Based on the literature and expert clinical opinion, Nee- 
dleman et al. identified 14 potential nursing-sensitive out- 
comes that could be measured using routinely collected 
administrative health data. These included: pressure 
ulcers, deep vein thrombosis and pulmonary embolism, 
pneumonia, urinary tract infection, central nervous system 
complications, shock or cardiac arrest, upper gastrointes- 
tinal bleed, pulmonary failure, physiologic/metabolic 
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derangement, surgical wound infection, mortality, failure 
to rescue and length of stay [9]. Administrative health data 
are electronic records collected for administrative pur- 
poses that include patients' hospital discharge summaries. 
These were determined to be the best source for con- 
structing nursing-sensitive outcomes, because they contain 
diagnoses and procedures coded according to the Inter- 
national Classification of Diseases (ICD) and contain pa- 
tient level variables such as age, sex, country of birth, and 
health insurance status in a relatively uniform format [9]. 
Needleman et al.'s development of nursing-sensitive out- 
comes was guided by three criteria: (1) that nursing- 
sensitive outcomes be conceptually related to nursing care, 
(2) that outcomes be 'codable' from hospital patient dis- 
charge (separation) abstracts, and (3) that the outcomes 
occur in inpatient acute care settings with high enough 
frequency and variation to allow for statistical analysis [9] 
[p 37]. 

For each outcome, an algorithm (syntax) was devel- 
oped that used a combination of ICD-9 codes, Diagnosis 
Related Groups (DRGs), Major Diagnostic Categories 
(MDCs), length of stay, presence of a surgical procedure 
and age. Each algorithm included outcome specific inclu- 
sion and exclusion (qualifying) criteria in an attempt to 
include only patients who experienced a truly prevent- 
able adverse outcome rather than one associated with 
the disease process. For example, the 'pressure ulcer' al- 
gorithm excluded patients with any form of paralysis in 
their hospital discharge records to ensure those flagged 
with a pressure ulcer were more likely to have acquired 
it as a result of the quality of nursing care and it was not 
present on admission as a consequence of a pre-morbid 
condition. Algorithms excluded outcomes that were pri- 
mary diagnoses, and used secondary diagnoses to iden- 
tify outcomes that were potentially nursing sensitive. As 
the algorithms developed by Needleman et al. used 
American ICD-9-CM (Clinical Modification) codes, sub- 
sequent translation to ICD- 10- AM (Australian Modifica- 
tion) codes using "crosswalks" (mapping keys) was 
undertaken by McCloskey [10] in New Zealand. These 
translated algorithms have been used in Australian stud- 
ies [11,12]. 

Many of the challenges faced by Needleman and his 
team [9] in the matching of data is obviated by the data 
linkage processes in Western Australia (WA), whereby 
all patients have a unique identifier that links their indi- 
vidual records into a single 'chain' of hospitalisation epi- 
sodes. Identification of comorbid conditions is enhanced 
because linked data allows researchers to match records 
of the same patient both within and across databases, 
thus providing longitudinal health data on individuals 
and populations [13-17]. Many patient hospitalisation 
databases only code patient conditions relevant to a spe- 
cific episode of care, therefore some chronic conditions 



may not be recorded in a discharge abstract [9,18]. An 
advantage of using linked data is being able to 'look back' 
to ascertain comorbid conditions for individuals when 
this is not routinely recorded. Similarly, linked data 
enables researchers to 'look forward' to determine pa- 
tient outcomes that require hospitalisation and are iden- 
tified following discharge from the index hospitalisation. 
The results of a WA study reported that identification of 
comorbid conditions in an adult population increased 
from 47% using index hospitalisation data to 90% when a 
three year look back into administrative health data was 
undertaken [17]. Therefore, linked data is likely to pro- 
vide more accurate information about comorbid condi- 
tions which may affect identification of nursing-sensitive 
outcomes. 

The WA Data Linkage Branch adheres to data linkage 
best practice protocols [19] and uses probabilistic match- 
ing based on medical record number, surname, first 
name and initial, date of birth, sex and address as the 
principal variables to link the data. Clerical review of 
additional information is undertaken for records that fall 
between definite matches and non-definite matches [20]. 
Invalid and missed links have been estimated at 0.11% 
[21]. Validation studies have shown recording of add- 
itional diagnoses and complications in Health Morbidity 
Data (HMD) vary between 10-80% sensitivity depending 
on the nature of the condition [21]. 

Studies of nursing-sensitive outcomes in administrative 
health data have predominantly used adult populations. 
Some researchers have included children within their adult 
study populations [6,22,23]; however, few have used admin- 
istrative health data of paediatric populations [24,25]. Given 
the differences in casemix and risk profiles between paediat- 
ric and adult patients [26,27], it is questionable whether 
nursing-sensitive outcomes used with adults are applicable 
to paediatrics. Optimal paediatric nursing care takes into ac- 
count the child's stage of development, consequently age 
stratification needs to be considered during analysis. Chil- 
dren have fewer chronic conditions and comorbidities and 
primary healthcare and in-home care delivery are empha- 
sised. As a result, paediatric hospital lengths of stay are 
shorter than those of adults, which contributes to the chal- 
lenges of studying paediatric populations. Earlier studies 
found that there were insufficient numbers of recorded 
events to make analysis meaningful for some potential 
nursing-sensitive outcomes [24,25]. Many comorbid condi- 
tions reported in adults are not present in children so the 
risks for developing adult nursing-sensitive outcomes are 
reduced. For the above reasons, it may not be useful to 
apply algorithms for nursing-sensitive outcomes that were 
validated in adult populations to paediatric populations. 
The algorithm for each outcome should be validated within 
paediatric populations before being used to measure the 
quality of paediatric nursing care. 
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In our earlier work with a panel of expert paediatric 
nurses, we established face and content validity for 17 
nursing-sensitive outcomes that were potentially useful 
for measuring the quality of paediatric nursing using ad- 
ministrative health data [28]. Seven of these outcomes 
were also used by Needleman et al., with pressure ulcer, 
pneumonia and surgical wound infection ranked as top 
three. The present study focused on these three nursing- 
sensitive outcomes and aimed to determine whether 
Needleman et al.'s algorithms, or variations of them, 
were useful in paediatric populations. A further aim was 
to determine whether linked data provided more accur- 
ate information about comorbid conditions than using 
data that was not linked, and whether it affected the 
identification of nursing-sensitive outcomes. 

Method 

This population based, retrospective cohort study used 
linked administrative health data from the WA HMD 
System which enabled ascertainment of all WA hospitali- 
sations for the cohort. The cohort included all WA resi- 
dent children admitted to one tertiary paediatric hospital 
during the 10 year period from July 1999 to June 2009 
inclusive. Inclusion criteria were that the child had a 
WA postcode as place of residence, was aged <18 years, 
and had been an inpatient (stayed at least one night; 
determined by admission date and separation date minus 
days of care provided by Hospital in the Home). Chil- 
dren who had been transferred from another hospital 
were excluded and data from subsequent hospitalisation 
following transfers were not included. Hospital separa- 
tions are only recorded when the child is discharged or 
transferred to another hospital. Separation records are 
not created when the child is transferred within the same 
hospital. To enable looking back for comorbid conditions 
and looking forward for possible consequences of the 
index hospitalisation, HMD from the previous ten years 
and all subsequent WA hospitalisations were provided 
for the children in the cohort. The index hospitalisation 
was the record in which the nursing-sensitive outcome 
was identified. 

The linked HMD provided abstracts of demographic 
and clinical information on hospital separations from all 
acute care hospitals within WA. Up to 22 diagnostic 
variables and 12 procedural variables were provided. 
Diagnoses were coded using ICD-9-CM until June 1999, 
and ICD-10-AM since then. Similarly, procedures were 
coded according to ICD-9-CM until June 1999, but sub- 
sequently followed the Australian Classification of 
Health Interventions (ACHI). Variables for MDC and 
Australian refined (AR)-DRG were also provided. 

Data were received in an anonymised file and analysed 
in SPSS for Windows (Version 19.0.0.1; 2010 IBM SPSS 
Chicago, II, USA), with logical checks undertaken during 



data clean-up. The translation of ICD, DRG and MDC 
codes was checked and amendments were made as 
required. The syntax for each nursing-sensitive outcome 
algorithm had been written for use in SPSS (Finn, J. un- 
published) and this was also checked and amended if ne- 
cessary. The MDC codes that were used were MDC4 
which are diseases and disorders of the respiratory sys- 
tem and MDC9 which are diseases and disorders of the 
skin, subcutaneous tissue and breast [18]. 

Month and year of birth were provided for each child. 
Age on admission was calculated, then categorised into 
developmentally appropriate age groups of neonate (1- 
28 days), infant (29-365 days), toddler (>l-3 years), pre- 
schooler (>3-6 years), school-age (>6-13 years) and ado- 
lescent (>13-<18 years). The risk pool for surgical 
wound infections was surgical patients only. This subset 
of the cohort contained patients classified as 'surgical' 
based on AR-DRG codes which were supplied by WA 
Data Linkage Branch. The remainder of the cohort was 
classified as 'non-surgical' and contained children coded 
as medical and 'other'. [18] 

Process 

Each hospital separation record was assumed to be an 
independent event, therefore calculations were based 
on hospital separations or records, rather than chil- 
dren, as done in other studies [7,11,24,29]. Nursing- 
sensitive outcomes of pressure ulcer, pneumonia and 
surgical wound infection were identified as per Needle- 
man et al.'s algorithms (Table 1). 

For each nursing sensitive outcome, frequencies of 
diagnostic and procedural codes listed in the index 
records of children with the outcome were calculated to 
identify any condition or procedure that might have con- 
tributed to the outcome, but had not been excluded in 
Needleman et al.'s algorithm. Selected index records were 
also examined individually along with previous records 
(look back) and subsequent separation records of the 
child to ascertain comorbid conditions which could in- 
crease risk. These comorbid conditions were then 
included as potential qualifying criteria in revised algo- 
rithms, which were applied to the paediatric population, 
and the index records of children determined to have the 
nursing-sensitive outcomes were identified. For example, 
to determine the effect of skin condition (MDC9) on the 
identification of children with nursing-sensitive pressure 
ulcers, the algorithm was run with and without the exclu- 
sion criterion of presence of a skin condition (MDC9). 

Each child's records were aggregated to ascertain the 
presence of a comorbid condition and, if present, check 
whether it was noted on the index record. Where neces- 
sary, the comorbid condition was added into the index 
record and algorithms for each nursing-sensitive out- 
come were re-run using the corrected index records. 
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Table 1 Definitions of nursing-sensitive outcomes 


Nursing-sensitive outcome 


Numerator 1 


Denominator 


Key exclusions 


As per Needleman et al. [9] 


Pressure ulcer 


Pressure ulcer (ICD 2 L89) 


All medical and 
surgical inpatients 
LOS > =4 days 


MDC9-skin conditions 
All diagnosis of hemiplegia, paraplegia, 
paralysis; cerebral palsy; (ICD>=G80 
and < =G84) 


Pneumonia 


Aspiration, post-operative, hypostatic, 

Kaftan 3 hrr™inf~nr~i anH i incneir~iriciH 
kJadtrl la I, Ul Ul Id IU al IU, Ul loUtrd ! IcU 

pneumonias: (ICD > =J14. 
and <=J15.6, J15.8 J15.9 J 1 8.0 
J18.2 J18.8 J18.9 J69.0 J95.8, J95.9) 


All medical and 

ci i rn i/"";a I i nnptipntc 
iu I y iL.a I M lua li ci I lo 


MDC4-respiratory conditions 

I m m i innrnrnnrnmicpH A 1 Pi's* 

I I II 1 1 1 Ul IUL.UI 1 1 Ul Ul 1 II ZJtTU, rll YJ S), 

(ICD > =B20. and < =B24, > = D80. 

and < =D89.9, M35.9) 

All diagnoses of probable community 

acquired pneumonia (ICD>=J10. 

diiG <. — j i u.o, > — j i i . anu <^ —j i i .o, —j i z. 

and < =J 1 2.9, J 1 3, J 1 5.7, J 1 6.8, > = J 1 7. 

and <=J 17.8) 


Postoperative pneumonia 
[24] 


Aspiration, post-operative, hypostatic, 
bacterial, viral, broncho pneumonias 
(ICD J12. and<=J12.9, J 13., >=J14. 

-, nr ] ^ — 1 1 c f, 1 1 c v 1 1 c o hen 
dl IU <L. — J 1 J.O, J ID./, J \ j.o, J \ j.y, 

> = J17. and < =J1 7.8, J 1 8.0, J18.1, 
J18.2, J18.8, J18.9, J69.0, J95.851, J95.9) 


All surgical inpatients 




Surgical wound infection 


Surgical wounds, including surgery 
post traumatic injury (ICD T79.3, 
T81.4 T81.41 T81.42) 


All surgical inpatients 
Total discharges only 




Recommended paediatric algorith 


m 






Pressure ulcer 


Pressure ulcer (ICD L89) 


All medical and 

ci irnii~a innptieintc 

jU 1 y |l_o 1 II IkJCl LI Cl 1 1 j 

LOS > =3 days 


All diagnosis of hemiplegia, paraplegia, 

nara k/cic i~cirf^Kra na cw cnina nifirla 
ljo i □ i y j i j, ccicljioi uciiiy, iun lo uniud. 

(ICD > =G80 and < =G84; > = Q05. 
and < =Q05.9 or > =Q07. and < =007.03) 
All diagnoses of paralysis found in 
look back period 


Pneumonia 


As per Needleman et al. plus 
J15.7, J18.1, J69.8 


All medical and 
surgical inpatients 


MDC4-respiratory conditions 
All diagnoses of probable community 
acquired pneumonia (ICD as per 
Needleman et al.) 

Aspiration pneumonia and epilepsy 
(ICD J69.0, 69.8, > = G40. and < =G40.9, 
> = G41. and<=G41.9) 
All diagnosis of epilepsy found in look 
back period 


Surgical wound infection 


Surgical wounds, including surgery 
post traumatic injury plus those 
found in 30 day look forward period' 
(ICDT79.3, T81.4T81.41 T81.42) 


All surgical inpatients 
Total discharges only 




1 All numerators are based on secondary diagnosis only except surgical wound infection in look forward period. 
2 ICD codes are all ICD-10-AM. 



Based on the results, algorithms were written that should 
optimise identification of nursing-sensitive outcomes in 
paediatric populations (Table 1). 

If a child had more than one type of nursing-sensitive 
outcome on the same or separate admissions they were 
counted as separate index hospitalisations. If a child was 
readmitted with the same nursing-sensitive outcome, the 
individual records were viewed. Clinical judgement was 
used to decide whether the outcome was the same epi- 
sode or an independent event based on time between 
discharge and admission, diagnoses and procedures. If 



judged to be independent events, records were counted 
as separate index hospitalisations and if judged to be the 
same episode, then only the first admission was included 
as the index hospitalisation. 

Statistical analysis 

Proportions of nursing-sensitive outcomes per 1,000 hos- 
pital separations were calculated for the three outcomes. 
Rates per 10,000 patient days were calculated using Nee- 
dleman's algorithms and the revised paediatric algo- 
rithms for pressure ulcer and pneumonia, but not 
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surgical wound infection. The definition of surgical 
wound infection used for both numerator and denomin- 
ator is 'within 30 days of surgery' [30] [p313]. It cannot 
be determined when the infection occurred in relation to 
the surgery from WA administrative health data nor can 
patients' lengths of stay post-surgery be ascertained. 
Therefore, a rate based on total patient days would not 
provide an accurate measure of the risk pool. 

Proportions and/ or rates for each nursing- sensitive 
outcome were determined by running the algorithm with 
the qualifying criterion included and then rerunning it 
with the qualifying criterion excluded. For example, the 
number of children with pressure ulcers who also had a 
form of paralysis and the number of children with pres- 
sure ulcer and no form of paralysis were determined. For 
each nursing-sensitive outcome a number of qualifying 
criteria were tested. Estimated rate differences and 95% 
confidence intervals (CI) were calculated. If there was no 
statistically significant difference in the rates (CI 
included zero) or the number of records was <5, the 
qualifying criterion was dropped from the algorithm. Fi- 
nally, percentages of records with comorbid conditions 
found for each nursing-sensitive outcome using look 
back data and nursing-sensitive outcomes using look for- 
ward data were calculated. 

Ethical considerations 

The project had approval from the Human Research Eth- 
ics Committees of the study hospital and the WA De- 
partment of Health. To avoid possible identification of 
children, wherever numbers were less than 5 the exact 
number is replaced by <5. 

Results 

From 1 July 1999 to 30 June 2009 a total of 129,719 hos- 
pital separations pertaining to 79,016 children met our 
inclusion/exclusion criteria. This increased to 517,605 
hospital separations when linked with records in the look 
back (back to 1 July 1989) and the look forward (to 30 
March 2010) periods. 

Of the 129,179 hospital separations, boys comprised 
57%, 32.4% were aged 6-13 years, 83% lived in the 
metropolitan area, 79% were 'emergency' admissions (as 
distinct from 'elective') and 83.4% were classified as 
'non-surgical'. The most common admission was respira- 
tory (16.5%), followed by gastrointestinal (12.8%) and 
musculoskeletal (12%) (Table 2). Excluding same-day 
admissions, the length of hospital stay (LOS) ranged 
from 1-975 days, with median 2 days (inter-quartile 
range 1-4 days). 

Pressure ulcers 

Adhering to the qualifying criteria used by Needleman 
et al. in Table 1 [9], over the 10 year study period, 49 



Page 5 of 12 



Table 2 Characteristics of the 10 year inpatient cohort 

Total hospital separations 
(n = 129719) 



n % 

Age group 

1 -28 days (neonate) 5650 4.4 

29-365 days (infant) 22114 17.0 

> 1-3 years (toddler) 23824 18.4 

>3-6 years (preschooler) 14786 11.4 

>6-1 3 years (school-age) 41988 32.4 

>1 3- < 18 years (adolescent) 21357 16.5 
Sex 

Male 73931 57.0 

Female 55788 43.0 
Residence 

Metropolitan 107715 83.0 

Rural 21908 16.9 
Admission type 

Emergency 102699 79.2 

Elective 27020 20.8 
Case mix 

Non-surgical 108127 83.4 

Surgical 21592 16.6 
Major diagnostic category 

Respiratory 21341 16.5 

Gastrointestinal 16593 12.8 

Musculoskeletal 15562 12.0 

Ear, nose throat and mouth 13326 10.3 

Neurological 10271 7.9 



hospital separation records included a code for pres- 
sure ulcer in one or more of the secondary diagnosis 
fields. This is equivalent to 1.39 pressure ulcers per 
1,000 hospital separations and a rate of 1.3/10,000 pa- 
tient days. 

Table 3 shows the proportions and rates of each quali- 
fying criterion for pressure ulcer. There were 2,751 
records of children with a form of paralysis of whom 
seven had pressure ulcers (2.54/1,000 separations; a rate 
of 4.75/10,000 patient days). Additionally, 126,968 hos- 
pital separations had no record of a form of paralysis, of 
which 60 had pressure ulcer as a secondary diagnosis 
(0.47/1,000 separations; a rate of 1.13/10,000 patient 
days). There was a significant difference between the 
rates of pressure ulcers when a child had a form of par- 
alysis recorded and when no paralysis was recorded (95% 
CI 1.83, 39.61). Numbers were too low for meaningful 
comparisons when the qualifying criteria of MDC9 (con- 
ditions of skin) were included or excluded (n < 5 pres- 
sure ulcers in children with a skin condition). Table 3 
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Table 3 Rates of pressure ulcers and analysis of qualifying criteria for pressure ulcer 


No. of events Total separations 


NO./1000 


Total patient 


Rate/10000 


Diff between rates/10000 




separations 


days 


patient days 


patient days 
95% CI 


As per Needleman 49 35125 


1.39 


376574 


1.30 




Qualifying criteria 










Paralysis 










With 7 2751 


2.54 


14727 


4.75 


(1.83, 39.61)* 


Without 60 126968 


0.47 


529376 


1.13 




MDC9 1 (disorders of skin or subcutaneous tissue) 










With <5 5225 


0.38 


17170 


1.16 


_2 


Without >62 124494 


0.52 


526933 


1.23 




LOS 3 










> = 2days 61 77471 


0.79 


491851 


1.24 


(0.45, 0.89)* 


<2 days 6 52248 


0.11 


52252 


1.15 




> = 3 days 57 52735 


1.08 


442379 


1.29 


(0.66, 1 .24)* 


<3days 10 76984 


0.13 


1 01 724 


0.98 




> = 4 days 53 37521 


1.41 


396737 


1.34 


(0.87, 1 .65)* 


<4days 14 92198 


0.15 


147366 


0.95 




As per recommended paediatric qualifiers 










No look back 54 51141 


1.05 


428598 


1.26 




With look back 25 49452 


0.51 


412555 


0.61 





1 Major Diagnostic Category. 

2 Insufficient number of events to compare rates. 

3 Length of stay. Categories are not exclusive. 
*p<.001. 



also provides details of numbers of hospital separations 
with pressure ulcers when length of stay was <2 or 
>2 days, <3 or >3 days and <4 or >4 days. The categor- 
ies are not exclusive and >4 days included those that are 
in >2 days length of stay. 

Spina bifida was identified as a risk factor when 
reviewing individual records of children who had a pres- 
sure ulcer. Therefore, the qualifying criterion in the 
paediatric algorithm that excluded children with paraly- 
sis in this study was revised to include ICD-10 codes for 
children with spina bifida (Table 1). 

The 10 year look back was used to determine whether 
any of the children with pressure ulcer(s) recorded as a 
secondary diagnosis had a comorbid condition, such as a 
form of paralysis that should have excluded their pres- 
sure ulcer from being considered nursing sensitive. Of 
the 67 separations with pressure ulcer diagnoses 
recorded, seven (10.4%) had codes for a form of paralysis 
on their index record, and a further 29 (43%) had a form 
of paralysis recorded in previous separation records only 
(Table 4). Incorporating look back resulted in the detec- 
tion of 0.51 nursing-sensitive pressure ulcers per 1,000 
hospital separations; a rate of 0.61/10,000 patient days 
(Table 3). 



Pneumonia 

Using Needleman et al.'s qualifying criteria [9] (Table 1), 
413 records of nursing-sensitive pneumonia (3.86 per 
1,000 hospital separations) were identified; a rate of 9.09/ 
10,000 patient days (Table 5). Similar to Table 3, Table 5 
shows the proportions and rates for various qualifying 
criteria. Numbers and rates of hospital separations with 
pneumonia are compared with and without cancer, im- 
mune deficiencies, community acquired pneumonia, re- 
spiratory system disorders and feeding difficulties. There 
were statistically significant differences between rates/ 
10,000 patient days for qualifying criteria of those with 
and without community acquired pneumonia (95% CI 
41.72, 96.06) and respiratory system disorders, coded 
MDC4 (95% CI 8.93, 15.48). There were <5 children 



Table 4 Paralysis identified in children with a pressure 
ulcer using a look back period 







No of events 


% of events 


Paralysis on 


same (index) separation 


7 


10.4 


Paralysis on 


previous separations only 


29 


43.3 


No paralysis 


recorded 


31 


46.3 
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Table 5 Rates of pneumonia and analysis of qualifying criteria for pneumonia 




No. of 


Total 


No./IOOO separations 


Total patient 


Rate/10000 


Diff between rates/10000 




events 


separations 




days 


patient days 


patient days 
95% CI 


As per Needleman 


413 


1 07026 


3.86 


454569 


9.09 




Qualifying criteria 














Cancer 














With 


11 


1972 


5.58 


13789 


7.98 


(-8.12, 1.46) 


Without 


600 


1 27747 


4.70 


530314 


11.31 




Immune deficiencies 














With 


<5 


357 


2.80 


2087 


4.79 


_i 


Without 


>606 


129362 


4.72 


542016 


11.25 




Secondary diagnosis of community acquired pneumonia 








With 


33 


620 


53.23 


4146 


79.59 


(41 .72, 96.06)* 


Without 


578 


1 29099 


4.48 


539957 


10.70 




MDC4 2 (disorders of respiratory system) 










With 


180 


21341 


8.43 


83481 


21.56 


(8.93, 1 5.48)* 


Without 


431 


108378 


3.98 


460622 


9.36 




Feeding difficulties 














With 


15 


2374 


6.32 


13716 


10.94 


(-5.90, 5.31) 


Without 


596 


1 27345 


4.68 


530387 


11.24 




Other pneumonia definitions 












Ventilator associated 


0 


2708 










Postoperative 


53 


21592 


2.45 


129275 


4.10 




Aspiration 


94 


129719 


0.72 


544103 


1.73 




Epilepsy 


26 


2306 


11.27 


9896 


26.27 


(14.90, 35.10)* 


No epilepsy 


68 


127413 


0.53 


534207 


1.27 




Feeding difficulty 


6 


2374 


2.53 


13716 


4.37 


(-0.80, 6.23) 


No feeding difficulty 


88 


1 27345 


0.69 


530387 


1.66 




As per recommended paediatric q 


ualifiers 










No look back 


368 


107141 


3.43 


455608 


8.08 




With look back 


337 


102877 


3.28 


436271 


7.72 





1 Insufficient number of events to compare rates. 

2 Major Diagnostic Category. 
*p<.001. 

with diagnoses of pneumonia and immune deficiency 
identified in the same separation (Table 5). 

Specific causes of pneumonia were analysed separately. 
There were no records of ventilator associated pneumonia 
(VAP) coded in any diagnostic categories despite 381,316 
hours of continuous ventilatory support being recorded in 
2,708 hospital separations. Rates of postoperative pneumo- 
nia [24] and aspiration pneumonia are shown in Table 5. 

Two groups of children at risk for aspiration pneumo- 
nia are those who have seizure activity or epilepsy and 
those with feeding difficulties [31]. There was a statisti- 
cally significant difference in rates of aspiration pneumo- 
nia between those who had epilepsy (26.27/10,000 
patient days) and those who did not (1.27/10,000 patient 



days) (difference 95% CI 14.90, 35.10). There were six 
children identified with aspiration pneumonia who had 
feeding difficulties, and the difference in rates was not 
significant (Table 5). Based on these results Needleman 
et al.'s algorithm was altered to include additional quali- 
fying criteria, and the revised paediatric algorithm 
(Table 1) identified 3.43 records of pneumonia per 1,000 
hospital separations; a rate of 8.08/10,000 patient days 
(Table 5). 

The 10 year look back was used to determine whether 
any of the children with pneumonia recorded as a sec- 
ondary diagnosis had a comorbid condition, such as the 
chronic respiratory conditions coded under MDC4, that 
should have excluded their pneumonia from being 
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considered nursing sensitive. One patient had cystic fi- 
brosis coded only on a previous separation and 13/611 
had asthma only coded in previous separation records 
accounting for 2% of events. Exclusion of records with 
MDC4 on the index record accounted for most of the 
chronic respiratory conditions therefore using look back 
for further chronic respiratory conditions was not 
included in the revised algorithm (Table 6). 

Data in the look back period were also reviewed for 
children with aspiration pneumonia recorded to ascer- 
tain whether previous diagnoses of epilepsy should have 
excluded their condition from being nursing sensitive. 
Thirty one hospital separations (33%) only recorded epi- 
lepsy prior to the index separation record of aspiration 
pneumonia (Table 6). Incorporating look back for epi- 
lepsy records resulted in the detection of 3.28 records of 
pneumonia per 1,000 hospital separations; a rate of 7.72/ 
10,000 patient days (Table 5). 



Table 7 Proportions of surgical wound infection and 
analysis of qualifying criteria for surgical wound infection 



No. of Total NO./1000 
events separations separations 



As per Needleman 


122 


21592 


5.65 


Qualifying criteria 








Cancer 








With 


0 


200 


0.00 


Without 


122 


21392 


5.70 


Immune deficiencies 1 








With 


0 


31 


0.00 


Without 


122 


21561 


5.66 


Readmitted with surgical 
wound infection 


103 


21470 


4.80 


As per recommended 
paediatric qualifier 


225 


21592 


1042 



1 Insufficient number of events to compare rates. 



Surgical wound infection 

There were 122 surgical wound infections identified in 
secondary diagnoses from 21,592 surgical hospital separ- 
ation records. The proportion of surgical wound infec- 
tions as per Needleman et al.'s algorithm was 5.65/1,000 
hospital separations. There were no exclusion criteria for 
this nursing-sensitive outcome in Needleman et al.'s al- 
gorithm. Assessment of whether having a compromised 
immune system or having cancer affected the number of 
events was undertaken, and indicated that no children 
who were immune compromised or had cancer had a 
surgical wound infection. This did not alter when data 
from the look back period was used (Table 7). 

Table 6 Comorbid conditions identified in children with 
pneumonia using a look back period 

No of events % of events 

Pneumonia 

Asthma (not coded as MDC4 1 ) on 
same separation 

Asthma (not coded as MDC4) on 
previous separations only 

No asthma or MDC4 recorded 

Cystic fibrosis (not coded MDC4) 
on same separation 



13 

592 
0 

1 



Cystic fibrosis (not coded MDC4) on 
previous separations only 

No cystic fibrosis or MDC4 recorded 61 0 

Aspiration pneumonia 

Epilepsy on same separation 26 
Epilepsy on previous separations only 31 
No epilepsy recorded 37 
1 Major Diagnostic Category, disorders of respiratory system. 



1.0 

2.1 

96.9 
0.0 

0.2 

99.8 

27.7 
33.0 
39.4 



A look forward period of 30 days from date of surgical 
admission was used to ascertain whether any children 
classified as surgical were readmitted within WA with a 
surgical wound infection as a primary diagnosis. There 
were 103 children (4.79/1,000 hospital separations) who 
were readmitted to a WA hospital with a surgical wound 
infection without a diagnosis of wound infection in their 
previous surgical separation record. This gave 10.42 
wound infections/1,000 surgical separations (Table 7). 

Discussion 

In contrast to other nursing-sensitive outcome studies 
using administrative health data [10-12,24,32], this study 
used linked data, which enabled the researchers to iden- 
tify all WA hospital separations during the previous 
10 years for every child in the study cohort. The advan- 
tage of this data linkage is that comorbid conditions that 
are not recorded in the same hospital separation as the 
one that contained the outcome of interest can be identi- 
fied. Using the linked data, 29 children with pressure 
ulcers (43% of all children with a pressure ulcer) were 
identified as also having a form of paralysis that was not 
identified in the same hospital separation as the pressure 
ulcer. Similarly, when using the look back period to iden- 
tify children who had any forms of epilepsy which 
increased aspiration pneumonia risk, a further 31 (33%) 
children with aspiration pneumonia were excluded from 
the risk set. When records of children identified with 
these comorbid conditions are excluded from analysis 
the proportions and incidence rates of the nursing- 
sensitive outcomes are reduced, and more accurately 
quantify outcomes that reflect the quality of nursing 
care. However, the risk of excluding some nursing- 
sensitive outcomes remains. 
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The standard for coding additional diagnoses in hos- 
pital morbidity data is that the diagnosis must impact 
on patient care that requires 'therapeutic treatment, 
diagnostic procedures or increased clinical care and/or 
monitoring' [18] [p 13]. Older children hospitalised for 
treatment not related to a comorbid condition, such as 
spina bifida, cerebral palsy or epilepsy, are often self- 
caring in relation to their chronic condition; they don't 
require an increase in resources, so these conditions 
are not coded in the index hospital separation abstract. 
The look back period in this study was 10 years pre- 
ceding each child's first separation from the tertiary 
hospital in the period 1999-2009, which for 77.3% of 
the children was from birth. Other researchers who 
have used linked data identified comorbid conditions 
either in the index separation record or they used a 
look back period, most frequently just 1 year [33]. For 
example, Preen et al. [17] used regression models to 
identify the impact of different comorbidity ascertain- 
ment look back periods and concluded that shorter 
periods of look back, approximately 1 year, were ap- 
propriate for post-hospitalisation mortality. However, it 
was suggested that longer look back periods were su- 
perior for other outcomes. The researchers reported 
that of the comorbid conditions recorded 5 years be- 
fore the index hospitalisation, 46.8% were recorded at 
the index hospitalisation. This increased to 68.6%, 
79.1% and 89.5% at 1, 2 and 3 years of look back re- 
spectively. The study was done with adult discharge 
abstracts and used 102 comorbid conditions identified 
in the Multipurpose Australian Comorbidity Scoring 
System (MACSS) [33]. Further work is required in this 
area, particularly in the paediatric population. 

Using linked data was also beneficial for looking forward 
to ascertain children's readmissions to hospitals in WA 
with a diagnosis of surgical wound infection. When the 
look forward period was included in the paediatric algo- 
rithm (Table 1), the proportion of surgical wound infec- 
tions nearly doubled. The rationale suggested by 
Needleman et al. for including surgical wound infection as 
a nursing-sensitive outcome was related to nurses' roles in 
preoperative preparation, which include skin cleaning and 
antibiotic administration [9] . However, the nurse's role also 
includes postoperative assessment and monitoring, which 
should lead to early intervention to prevent wound infec- 
tion. A principle of current paediatric care encourages 
home care as much as possible and results in shorter 
lengths of stay following surgery [34,35]. As surgical 
wound infections may not be noticed until a child has 
been discharged, it is reasonable to consider that children 
who present with a wound infection within a 30 day period 
of discharge from a surgical admission have an infection 
that is potentially nursing sensitive. Occurrence of wound 
infection could also reflect poor discharge planning with 



the child and their family regarding post-hospital wound 
care and medications. If parents are concerned about pos- 
sible surgical wound infection in their child, children may 
return to the outpatient department, the surgeon's private 
clinic, a general practitioner or emergency department and 
not require hospital admission with a surgical wound in- 
fection. These children are not accounted for in the 
current data linkage therefore rates are probably under- 
estimating the true occurrence of infection. 

Indicators of the quality of care may be used to compare 
rates of events across time or within and between units 
and hospitals. We were unable to ascertain whether the 
additional records with look back and look forward 
comorbid conditions were randomly distributed between 
hospitals or over time. If they were, calculations of rates 
would have consistent errors so including or excluding the 
extra cases would make little difference on relative per- 
formance when used for benchmarking. 

As well as amending Needleman et al.'s algorithms to in- 
corporate look back data for certain comorbid conditions, 
and look forward data for surgical wound infections, the 
algorithms were further tailored to suit paediatric popula- 
tions by incorporating variations of qualifying criteria 
based on their impact in the study population. Differences 
between proportions or rates of events were analysed with 
and without each qualifying criterion applied for each 
nursing-sensitive outcome. For pressure ulcers statistically 
significant differences were found for each qualifier, except 
those with MDC9 (diseases of skin and subcutaneous tis- 
sues), where numbers were too low for meaningful ana- 
lysis. Similarly, for pneumonia and surgical wound 
infection, numbers with immune deficiencies were too 
low; and for surgical wound infection, numbers with can- 
cer were also too low for meaningful analysis. Although 
these qualifying criteria have not been included in our 
paediatric algorithms, regular reviews are recommended 
as the numbers of children with these diagnoses will alter 
with changes in treatments, diseases and coding. 

Length of stay was analysed to ascertain the most appro- 
priate duration qualifying criterion to apply when deter- 
mining hospital-acquired pressure ulcers in children. 
There is a lack of consistency between researchers: some 
include adult hospital length of stay of longer than 3 days 
[9], others use longer than 4 days [36-39]. Curley et al. 
[40] found evidence of hospital acquired pressure ulcers in 
children as early as the second day of their hospital stay 
when assessing children in intensive care units during a 
point prevalence study. Number of pressure ulcers was 
low in our study when the length of stay criterion was less 
than 2 days, versus 2 or more days, so the statistical signifi- 
cance of the differences in rates using this criterion should 
be interpreted with caution. However, it is recommended 
that the paediatric algorithm for pressure ulcer include 
children who had a length of stay of longer than two days. 
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Patients with cancer are often immunocompromised 
and at increased risk of developing infections. Therefore, 
algorithms for nursing-sensitive outcomes that are the re- 
sult of infections often include a qualifying criterion that 
removes this high risk group. The difference in rates of 
pneumonia in children with and without cancers was not 
statistically significant. As the rate was higher in children 
without cancer than those with cancer, this qualifying cri- 
terion was not included in the paediatric algorithm. Nee- 
dleman et al. also removed it from their algorithm as it 
did not enhance the specificity of identifying outcomes in 
adult populations [9]. On the other hand, the criterion 
has been retained by others when calculating incidence 
of outcomes that are hospital acquired infections based 
on clinical reasoning [38,39]. The latest version of out- 
comes recommended by the Agency for Healthcare Re- 
search Quality (AHRQ) has not excluded children with 
cancers but stratified them into different risk groups [41]. 
In larger populations the number of children with cancer 
and pneumonia may be greater and stratification or ex- 
clusion of this group of children may be necessary. 

As pneumonia is frequentiy used to indicate the quality of 
nursing care, [7,24,42] and the results of our earlier Delphi 
study [28] prioritised subgroups of pneumonia as nursing- 
sensitive outcomes, proportions and rates of three sub- 
groups were calculated. The ICD codes for the individual 
pneumonia types were included in Needleman et al.s algo- 
rithm. Ventilator associated pneumonia, which is particu- 
larly used to indicate the quality of care in intensive care 
settings, had no records coded despite children in the cohort 
requiring 381,316 hours of ventilation. Follow up with the 
coders confirmed that there were no records coded as they 
did not identify any diagnoses of ventilator associated pneu- 
monia in the medical records (Logan, J. personal communi- 
cation). Postoperative pneumonia, which is applicable to 
surgical patients only [24], appears to be a viable indicator 
of the quality of nursing care, but there were too few records 
of aspiration pneumonia in our cohort to evaluate it. 

This cohort of hospitalised children is similar to cohorts 
of children in public hospitals across Australia. More boys 
than girls under 15 years of age are hospitalised [43] and 
the most common reasons for hospital admission are re- 
spiratory and gastrointestinal conditions in children aged 
less than 14 years, and rates of injury, poisoning and other 
external causes increase with age [44,45]. Other leading 
causes of hospitalisation are chronic diseases of tonsils and 
adenoids [45]. 

Limitations 

A limitation of using WA administrative health data for 
identifying nursing sensitive outcomes is the inability to 
determine whether secondary diagnoses are pre-existing 
comorbid conditions or complications that occurred dur- 
ing hospitalisation. The use of linked data can reduce 



this limitation by identifying children who have comor- 
bid conditions recorded in previous separation records 
and can be excluded from being considered to have a 
nursing-sensitive outcome. 

A further limitation is that the date of surgery was not 
included in HMD, so the 30 days post-surgery timeframe 
assumed that surgery occurred on day one of the hospital- 
isation. There is potential to have under estimated the rate 
of surgical wound infections as patients who had their sur- 
gery later than day one would not have had 30 days post- 
surgery included in the analysis. However, it may be more 
accurate than no outer time limit. [46,47] To increase the 
accuracy of this indicator, and others which measure qual- 
ity postoperatively, it would be beneficial if the date of sur- 
gery became a routine variable in HMD. 

There is a likelihood of under or over reporting of out- 
comes. Under reporting occurs particularly when the 
outcomes have no financial implication for the health 
service provider [9,48]. Not all nursing care is recorded 
in patient's clinical records [25,27] and not all of the clin- 
ical records are coded into the separation records [18]. 
However, an audit of coding of ICD-10-AM showed a 
high level of reliability and adherence to coding stan- 
dards [49]. Validation studies using review of hospital 
charts have found a reasonably high level of data accur- 
acy and reported 87% accuracy for DRG coding within 
WA HMD [21]. The advantages of feasibility, cost saving 
and having complete longitudinal population data when 
using administrative health data offsets the limitations 
and can provide reliable population-based estimates of 
nursing-sensitive outcomes. 

As numbers of nursing-sensitive outcomes are small in 
this WA paediatric population, comparisons of differences 
in rates may not be reliable, particularly when there are 
fewer than 5 records [50]. It is important to consider the 
number of records when interpreting results. 

Implications for research and policy 

Using linked data is advantageous for identifying comor- 
bidities that are not recorded on index separation records, 
but exclude children from being considered to have 
nursing-sensitive outcomes. When comorbidity records 
from previous separations are included, the specificity of 
the algorithm is increased, and a more accurate number of 
actual nursing-sensitive outcomes can be ascertained. Fur- 
thermore, linked data assists in identifying outcomes that 
become evident following discharge. When using adminis- 
trative heath data, linked data should be used, particularly 
if all comorbid conditions are not routinely identified in 
the index separation record and to assist in the accurate 
identification of outcomes. 

Numbers of pressure ulcers recorded in administrative 
health data in children are too few to be a useful meas- 
ure of the quality of nursing care in this population. This 
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finding is similar to results of a study undertaken in a 
large paediatric cohort in California [24]. However, the 
algorithms for pneumonia and surgical wound infections 
could potentially be considered for quality improvement 
initiatives within a hospital. As the health system and paedi- 
atric cohorts are similar across Australia [51], these algo- 
rithms are a starting point for national benchmarking to 
compare events and rates within tertiary paediatric hospi- 
tals. However, before being used for benchmarking between 
hospitals or areas within a hospital, additional risk factors 
must be identified to allow appropriate stratified analysis. 
Furthermore, paediatric algorithms can be used to deter- 
mine whether there are associations between nursing- 
sensitive outcomes and levels of nurse staffing that would 
assist in ascertaining appropriate staffing levels in paediatric 
hospitals. 

Conclusion 

Validation of algorithms prior to their use is an import- 
ant step in the process of measuring nursing- sensitive 
outcomes in different patient populations. Changes need 
to be made to adult algorithms before applying them to 
paediatric cohorts. Using linked data is advantageous in 
enhancing the sensitivity of algorithms for nursing- 
sensitive outcomes. 
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